Add OpenTelemetry v2 integration with enhanced features and comprehensive testing by tconley1428 · Pull Request #1314 · temporalio/sdk-python

tconley1428 · 2026-02-03T01:39:24Z

Summary

This PR introduces a new OpenTelemetry v2 integration for the Temporal Python SDK with significant enhancements over the existing OpenTelemetry support. The integration provides deterministic tracing, comprehensive test coverage, and improved maintainability.

Key Features Added:

Deterministic ID generation: Uses Temporal's workflow-safe random generator for consistent span/trace IDs across replays
Comprehensive operation support: Full tracing for workflows, activities, local activities, child workflows, timers, signals, updates, queries, and Nexus operations
Context propagation: Proper trace context propagation across all Temporal operation boundaries
Plugin architecture: Clean plugin-based integration using Temporal's SimplePlugin base class
Workflow-safe span creation: temporalio.contrib.opentelemetryv2.workflow.start_as_current_span() for user workflow tracing

Architecture Improvements:

Modular design: Restructured as proper package with separate modules for interceptor, plugin, processor, and workflow utilities
Replay-aware processing: TemporalSpanProcessor skips span export during workflow replay to prevent duplicate telemetry
Read-only mode detection: Added workflow.unsafe.is_read_only() to handle queries and update validators safely
Enhanced interceptor: Comprehensive TracingInterceptor covering all client and worker operations

Testing & Quality:

Comprehensive test suite: Added test_opentelemetryv2_comprehensive_tracing covering all workflow operations with proper span hierarchy validation
Test isolation: Pytest fixtures ensure OpenTelemetry state doesn't leak between tests
Span hierarchy validation: Tests use dump_spans() for maintainable hierarchy validation similar to existing OpenTelemetry tests
Complete linting compliance: All code passes ruff, pyright, mypy, and pydocstyle checks with comprehensive docstrings

Test plan

All existing tests continue to pass
New comprehensive test validates tracing across all Temporal operations
Span hierarchy validation ensures proper parent-child relationships
Test isolation prevents OpenTelemetry state conflicts between test runs
Integration works with both basic tracing (add_temporal_spans=False) and comprehensive tracing (add_temporal_spans=True)
Deterministic span IDs maintained across workflow replays
No span export during workflow replay to prevent duplicate telemetry
Read-only operations (queries, update validators) handle tracing safely
All linting and type checking passes

🤖 Generated with Claude Code

This commit adds a new OpenTelemetry interceptor (opentelemetryv2) with enhanced capabilities for Temporal workflow integration: Features: - Deterministic ID generation for spans/traces in workflows using TemporalIdGenerator - Context propagation across workflow and activity boundaries - Support for workflow-level span creation via workflow.start_as_current_span - Enhanced interceptor with context propagation to activities and nexus operations - Compatible with existing opentelemetry module while providing additional functionality Implementation: - New TemporalIdGenerator uses workflow.random() for deterministic IDs in workflows - TracingInterceptor handles client, worker, activity, workflow, and nexus operations - Workflow-safe span creation context manager in workflow module - Comprehensive test coverage for trace propagation scenarios This is separate from the OpenAI agents OTEL integration and provides general-purpose OpenTelemetry improvements for Temporal workflows. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

…inting fixes This commit significantly improves the OpenTelemetry v2 integration for the Temporal SDK with the following enhancements: ## Core Features Added: - **Comprehensive test coverage**: Added `test_opentelemetryv2_comprehensive_tracing` covering all workflow operations including activities, local activities, child workflows, timers, signals, updates, queries, and Nexus operations - **Read-only mode detection**: Implemented `workflow.unsafe.is_read_only()` to prevent span ID generation errors during queries and update validators - **Test isolation**: Added pytest fixture to reset OpenTelemetry tracer provider state between test runs - **Span hierarchy validation**: Refactored tests to use `dump_spans()` hierarchy validation for better maintainability ## Linting and Documentation: - Fixed all import path issues for OpenTelemetry ID generators - Added comprehensive docstrings for all public classes and methods - Fixed type annotations and null handling throughout the codebase - Resolved Nexus headers access issues with proper type protocols - Achieved complete pydocstyle compliance ## Technical Improvements: - Enhanced `TemporalSpanProcessor` with proper replay handling - Improved `TemporalIdGenerator` with deterministic workflow-safe random generation - Updated span parenting validation to ensure proper trace relationships - Added max_cached_workflows=0 to all test workers for deterministic behavior 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

…vements

temporalio/contrib/opentelemetry/__init__.py

temporalio/contrib/opentelemetry/_id_generator.py

temporalio/contrib/opentelemetry/_otel_interceptor.py

temporalio/contrib/opentelemetry/_interceptor_v2.py

temporalio/contrib/opentelemetry/_plugin.py

temporalio/contrib/opentelemetry/_processor.py

…essor

temporalio/contrib/opentelemetry/_id_generator.py

temporalio/contrib/opentelemetry/_otel_interceptor.py

temporalio/contrib/opentelemetry/_interceptor_v2.py

temporalio/contrib/opentelemetry/_plugin.py

temporalio/contrib/opentelemetry/_processor.py

temporalio/contrib/opentelemetry/_tracer_provider.py

…rovider in a workflow will use a replay safe version. Care should still be taken if creating one from scratch inside a workflow

…vements

cretz

Just a couple of minor things now...

temporalio/contrib/opentelemetry/_otel_interceptor.py

temporalio/contrib/opentelemetry/_plugin.py

temporalio/contrib/opentelemetry/_tracer_provider.py

temporalio/plugin.py

…uring replay

cretz

This looks good to me, unsure if we want @THardy98 to look as well. I am comforted by the fact that this is opt-in and marked experimental.

…nterceptors if not present in the worker's client

THardy98

walked through it with tim lgtm

cretz · 2026-02-13T15:13:57Z

temporalio/plugin.py

-        if interceptors is not None:
-            config["interceptors"] = interceptors
+
+        # Only propagate client interceptors if they are provided as a simple list (not callable)


I don't think you should propagate client plugins to workers, that is what our system does. Plugins should just provide the values that would be set for those two objects IMO. But we can discuss this. Maybe we just need a general interceptors option on simple plugin that is smart enough to check interceptor type and prevent double-register?

I don't think you should propagate client plugins to workers, that is what our system does.

If you register it on the client, which isn't guaranteed.

Maybe we just need a general interceptors option on simple plugin that is smart enough to check interceptor type and prevent double-register?

Maybe. With this change that's functionally the same as just providing them to client_interceptors, but might be less confusing for implementors.

tconley1428 and others added 2 commits January 20, 2026 12:38

tconley1428 requested a review from a team as a code owner February 3, 2026 01:39

tconley1428 added 7 commits February 4, 2026 14:19

Add debugging to comprehensive test

066438b

Fix formatting

68f17fd

Skip test on timeskipping

fa25bbc

Merge opentelemetry contribs

809b22d

Switch to batch processor

0518e77

Merge remote-tracking branch 'origin/main' into opentelemetryv2-impro…

15e77b4

…vements

Use new workflow random functionality for id generation

a838aeb

cretz reviewed Feb 6, 2026

View reviewed changes

tconley1428 added 4 commits February 6, 2026 11:04

Update to remove global state modification from plugin, and span proc…

0628859

…essor

Remove inaccurate comment

94eda7a

Move otel tests

ae1e7f5

Fix test import

3678a18

cretz reviewed Feb 9, 2026

View reviewed changes

tconley1428 added 12 commits February 9, 2026 16:37

PR Feedback

b93af2d

Rely on the global and pass it through. All uses of a global tracer p…

9fc296d

…rovider in a workflow will use a replay safe version. Care should still be taken if creating one from scratch inside a workflow

Merge remote-tracking branch 'origin/main' into opentelemetryv2-impro…

1e675ef

…vements

Fix rebase issue with standalone activities context

92aa182

Clean up some unused code paths

ce3e421

Change tracerprovider return type

9db654b

Remove is read only

f4117e5

Clean up test prints

8416666

Return ReplaySafeTracerProvider

de676c5

Fix lint

bc8eb16

Add readme

f01c03d

Debugging

d25cb3f

cretz reviewed Feb 11, 2026

View reviewed changes

More debugging

e0f26a2

tconley1428 added 5 commits February 11, 2026 11:27

More debugging logs

de5a329

Change debugging

efda2f0

Switch to is_read_only because updatevalidators are not technically d…

d6e9442

…uring replay

Linting

af2d637

Remove interceptor tracer members

d1d6630

cretz approved these changes Feb 12, 2026

View reviewed changes

tconley1428 and others added 4 commits February 12, 2026 09:55

Change plugin interceptor logic to propagate client/worker combined i…

d81d859

…nterceptors if not present in the worker's client

Pass through all of otel

0cb5267

Remove debug file

222c503

Merge branch 'main' into opentelemetryv2-improvements

9a1b79d

THardy98 approved these changes Feb 12, 2026

View reviewed changes

cretz reviewed Feb 13, 2026

View reviewed changes

tconley1428 merged commit b69ac92 into main Feb 13, 2026
25 of 27 checks passed

tconley1428 deleted the opentelemetryv2-improvements branch February 13, 2026 16:39

Conversation

tconley1428 commented Feb 3, 2026

Summary

Key Features Added:

Architecture Improvements:

Testing & Quality:

Test plan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cretz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cretz left a comment

Choose a reason for hiding this comment

Uh oh!

THardy98 left a comment

Choose a reason for hiding this comment

Uh oh!

cretz Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

tconley1428 Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tconley1428 Feb 13, 2026 •

edited

Loading